Developing Handwriting-based Intelligent Tutors to Enhance Mathematics Learning

Mathematics is a topic in American education in which students lag behind their international peers, yet it is a key building block for high-performing careers in science, computers, and engineering. Intelligent tutoring systems have been helping to narrow this gap by providing students with opportunities to practice problem-solving and receive detailed feedback along the way, letting them work at their own pace and practice specific concepts. Prior to this work, intelligent tutors for math have been shown to improve student performance one standard-deviation above traditional classroom instruction [35]. This dissertation explores ways to improve this effect via the use of alternative input modalities, specifically: handwriting input, and investigates the impact on learning in the domain of algebra equation solving. This dissertation shows that handwriting provides usability benefits in that speed of entry increases, user error decreases, and user satisfaction increases. Furthermore, it shows that handwriting may also provide learning benefits: students solving the same problems by handwriting as others who are typing experience a faster learning rate. Specific math advantages of using handwriting are: a reduction in extraneous cognitive load due to the affordance of handwriting for more direct manipulation, and improved support for the two-dimensional spatial information which is inherently meaningful in mathematics (e.g., vertical fraction notation). This dissertation investigates these factors and their impact. One concern with the use of handwriting in intelligent tutoring systems, however, is that recognition technology is not perfect. To the extent that the system cannot be confident of correctly recognizing what the student is writing, it cannot identify tutoring opportunities and provide detailed, step-targeted feedback. Therefore, a trade-off is clear between the difficulty of improving recognition accuracy and the need to support step-targeted feedback. One strategy to address this trade-off is using a type of instruction based on worked examples, which provide a sort of feed-forward to guide learners. A second strategy is to investigate technical approaches to improving handwriting recognition accuracy. This dissertation explores two methods of enhancing baseline recognition: training the recognition engine on a data corpus of student writing in order to maximize writerindependent recognition accuracy; and making use of domain-specific context information on the fly to refine the recognition hypotheses. The approach taken in this dissertation includes technical development, pedagogical development, and user studies. Topics addressed include what the advantages of using handwriting are, how the above factors contribute to these advantages, and how these advantages can be leveraged in real tutoring systems. Reasonable writer-independent handwriting recognition rates can be achieved by a priori data collection and training, and these can be even further improved via the addition of domain-context information. Furthermore, a realistic tutoring interaction paradigm can be achieved through these methods, in spite of imperfect raw recognition accuracy. This dissertation leaves the door open to continued work on basic recognition technology which can improve the achievements reported here even further.